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■ Abstract 
O . 

' It is well-known that the representation theory of the finite group of unipotent upper- 

, triangular matrices J7„ over a finite field is a wild problem. By instead considering approxi- 

mately irreducible representations (supercharacters), one obtains a rich combinatorial theory 
analogous to that of the symmetric group, where we replace partition combinatorics with 
set-partitions. This paper studies the supercharacter theory of a family of subgroups that 
- , interpolate between C/„_i and Un- We supply several combinatorial indexing sets for the 

Pm ' supercharacters, supercharacter formulas for these indexing sets, and a combinatorial rule 

, for restricting supercharacters from one group to another. A consequence of this analysis is a 

Pieri-like restriction rule from Un to Un-i that can be described on set-partitions (analogous 
to the corresponding symmetric group rule on partitions). 



1 Introduction 

>. 

I The representation theory of the finite group of upper-triangular matrices [/„ is a well- 

ed . known wild problem. Therefore, it came as somewhat of a surprise when C. Andre was 

able to show that by merely "clumping" together some of the conjugacy classes and some of 
the irreducible representations one attains a workable approximation to the representation 
I theory of f/„ [H 1^1 [31 H] • In his Ph.D. thesis [2], N. Yan showed how the algebraic geometry 

of the original construction could be replaced by more elementary constructions. E. Arias- 
I Castro, P. Diaconis, and R. Stanley 8: were then able to demonstrate that this theory can 

in fact be used to study random walks on C/„ using techniques that traditionally required 
the knowledge of the full character theory [11]. Thus, the approximation is fine enough 
■ to be useful, but coarse enough to be computable. Furthermore, this approximation has 

^ I a remarkable combinatorial structure analogous to that of the symmetric group, where we 

replace partitions with set-partitions. 

Almost irreducible 1 J Labeled set partitions 

representations of C/„ J |^ of {1, 2, . . . , n} 

One of the main results of this paper is to extend the analogy with the symmetric group by 
giving a combinatorial Pieri-like formula for set-partitions that corresponds to restriction in 

In [To], P. Diaconis and M. Isaacs generalized this approximating approach to develop a 
the concept of a supercharacter theory for all finite groups, where irreducible characters are 
replaced by supercharacters and conjugacy classes are replaced by superclasses. In particular, 
their paper generalized Andre's original construction by giving an example of a supercharac- 
ter theory for a family of groups called algebra groups. For this family of groups, they show 
that supercharacters restrict to Z>o-linear combination of supercharacters, tensor products 
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of supercharacters are Z>o-lincar combinations of supercharacters, and they develop a notion 
of superinduction that is the adjoint functor to restriction for supercharacters. This paper 
uses a family of subgroups that interpolate between Un and Un-i to explicitly decompose 
restricted supercharacters from C/„ to Un~i- 

Section 2 reviews the basics of supercharacter theory and pattern groups. Section 3 
defines the interpolating subgroups ?7(„i) , and finds two different sets of natural superclass and 
supercharacter representatives, which we call comb representatives and path representatives. 
Section 4 uses a general character formula from [fS] to determine character formulas for both 
comb and path representatives. The character formula for comb representatives - Theorem 
14.11 - is easier to compute directly, but the path representative character formula - Theorem 
14.31 - has a more pleasing combinatorial structure. Section 5 uses the character formulas 
to derive a restriction rule for the interpolating subgroups given in Theorem 15. II Corollary 
[SH] iterates these restrictions to deduce a recursive decomposition formula for the restriction 
from Un to Un-l- 

This paper is the companion paper to ^3, , which studies the superinduction of super- 
characters. Other work related to supercharacter theory of unipotent groups, include C. 
Andre and A. Neto's exploration of supercharacter theories for unipotent groups of Lie types 
B, C, and D [5], C. Andre and A. Nicolas' analysis of supertheories over other rings [B], 
and an intriguing possible connection between supercharacter theories and Boyarchenko and 
Drinfeld's work on L-packets H]. 

Acknowledgements 
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2 Preliminaries 

This section gives reviews several topics fundamental to our main results: Supercharacter 
theories, pattern groups, and a character formula for pattern groups. 

2.1 Supertheories 

Let G be a group. A supercharacter theory for G is a partition 5^ of the elements of G and 
a set of characters S, such that 



where (, ) is the usual innerproduct on class functions, 

(d) Every % G 5 is constant on the elements of . 

We call 5^ the set of superclasses and S the set of supercharacters. Note that every group 
has two trivial supercharacter theories - the usual character theory and the supercharacter 
theory with 5^ = {{1},G \ {1}} and S = {1,7g ^ l}, where 1 is the trivial character of G 
and 7g is the regular character. 

There are many ways to construct supercharacter theories, but this paper will study a 
particular version developed in [10] to generalize Andre's original construction to a larger 
family of groups called algebra groups. 



(a) 
(b) 
(c) 




(7,X) >0, 
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2.2 Pattern groups 

While many results can be stated in the generality of algebra groups, frequently statements 
become simpler if we restrict our attention to a subfamily called pattern groups. 

Let Un denote the set of n x n unipotent upper-triangular matrices with entries in the 
finite field ¥q of q elements. For any poset V on the set {1,2,..., n}, the pattern group U-p 
is given by 

U-p = {u € Un I Uij ^ implies i<jmV}. 

This family of groups includes unipotent radicals of rational parabolic subgroups of the 
finite general linear groups, and f/„ is the pattern group corresponding to the total order 
l<2<3<-<n. 

The group U-p acts on the F^-algcbra 

np = {u - 1 I M G Up} 

by left and right multiplication. Two elements u,v € Up are in the same superclass if 
u — 1 and V — 1 are in the same two-sided orbit of rip. Note that since every element of 
Up can be decomposed as a product of elementary matrices, every clement in the orbit 
containing v — 1 € np can be obtained by applying a sequence of the following row and 
column operations. 

(a) A scalar multiple of row j may be added to row i if j >i'mV, 

(b) A scalar multiple of column k may be added to column li{k<l'mV. 
There are also left and right actions of Up on the dual space 

tip = {A : np — »• Fg | A Fg-linear}, 

given by 

{uXv){x — 1) = X{x~^{x — l)y~^), where A G n^, u,v,x G Up. 

Fix a nontrivial group homomorphism : F+ ^ C^. The supercharacter with represen- 
tative A e Up is 

\Uv^Uv\ ^^^^^ 

We many identify the functions A £ with matrices by the convention 

Xij = A(e,j), where G rip has entry 1 and zeroes elsewhere. 

Then, as with superclasses, every element in the orbit containing A G can be obtained by 
applying a sequence of the following row and column operations. 

(a) A scalar multiple of row i may be added to row ji{i<j'mV, 

(b) A scalar multiple of column I may be added to column k if I > k inV. 

Note that we ignore (or set to zero) all nonzero elements that might occur through these 
operations that are not in allowable in n^. For example, if i is incomparable to j in V 
and a row operation would cause Xij ^ 0, we carry through the operation as usual, and 
retroactively set Xij back to zero (since A does not "see" these entries). 

Exeimple. For Un we have 

f Superclasses 1 f < u — 1 has at most one nonzero ^ (2 1) 

\ ofUn J I " ^ I element in every and column J \ • / 



Note that if g' = 2, then 

\u€Un 



u — 1 has at most one nonzero 1 j Set partitions 
element in every and column J 1 of{l,2,...,n} 
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Similarly, if 



n„ = Un-l 

n* = {A : n„ ^ Fg | A F^-linear}. 



then 



Let 



Supercharacters 

oiUn 



A e n! 



The matrix A has at most one non- 
zero element in every and column 



for {ij),{kj) e J. 



a tracks occurrences of 

b tracks occurrences of 
M tracks occurrences of 



Ajfe 








Uij 


Aifc 








Ujk 





(2.2) 



Sn{q) = {A G n* I A has at most one nonzero element in every and column}. (2.3) 

2.3 A supercharacter formula for pattern groups 

Let U-p be a pattern group with corresponding nilpotent algebra n-p. Let 

J = {{i,j) I i<jinr}. 
Given u ^ U-p and A G n^, define a, 6 G ¥q^^ by 

Uij = ^ UjkXtk, for G J, 

j<k in V 

^jk= X! for(j, fc)GJ. 

i<j in V 

Let M be the | J| x | J| matrix given by 

_ J UjkKu ifi<j<k<lmV, 
rnjMi otherwise. 

Informally, if one superimposes the matrices u and A, then 



The following theorem gives a general supercharacter formula for pattern groups. How- 
ever, to properly use the theorem we will need to choose appropriate superclass and super- 
character representatives. 

Theorem 2.1 ([12]). Let u e Up and Xen^. Then 
(a) The character 

xHu) = 

unless there exists x G Fg''' such that Mx = —a and b 1. Null(il/), 
//X'^(w) ™i zero, then 

„|C/pA| 



{x-b)eoX{u-l), 



where ■ is the usual inner product (dot product) on Fg^'. 
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Remark. There are two natural choices for x^: one of which is the conjugate of the other. 
Theorem 12.11 uses the convention of [1^ rather than [12] . 

C. Andre proved the [/„-version of this supercharacter formula for large characteristic, 
and [5] extended it to all finite fields. Note that the following theorem follows from Theorem 
I2.1l bv choosing appropriate representatives for the superclasses and supercharacters. 

Theorem 2.2. Let A G Sn{q), and let u d Un be a superclass representative as in \2.1]) . 
Then 



(a) The character degree 



(h) The character 



X\u) = 



unless whenever Ujk ^ with j < k, we have Xij = for all i < j and Xji = for all 



I > k. 

(c) lfx^{u)^0, then 



x^{i)eoX{u-i) 



q\{i<3<k<l I Ujk,^il€rq }\ 

3 Interpolating between [/„ i and Un 

Fix 71 > 1. For < m < n, let 



n 
I 



U(,n) = {u eUn I uij ^ 0, for j <m} = U-Pf^ 

"(m) = {u-1 I U e [/(,„)} = n-p,^) , 



= {A : n 



(m) 



A Fg-lincar} = 



where = 1 



TO + 1 



TO — 1 
I 



Note that 

Un-l = C/(n) <1 [/(„-!) <■ ■■< U^i) < J7(o) = Un- 

The goal of this section is to identify suitable orbit representatives for representatives for 
U{m)\n(m)/U[m) and C/(j„)\n^^^ /[/(„). 

A matrix A has an underlying vertex-labeled graph structure Ga given by vertices 

Va = {Nonzero entries in A} 

and an edge from Aij to A^i if i = k or j = I. We label each vertex by its location in the 
matrix, so Ay has label {i, j). For example, for a, b, c, d, e, f,g,h£ ¥^ , 



A 



/ a 6 \ 

c d 

e / 

.9 

V /i y 



implies Ga 
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3.1 Superclass representatives 

Unlike with C/„, the interpolating groups have several natural representatives to choose 
from. In this case, we consider a "natural choice" of an orbit representative to be one with 
a minimal number of nonzero entries. This section introduces two particular examples. 
A matrix u S U^m) is a comb representative if 

(a) At most one connected component of Gu-i has more than one element, 

(b) If Gu-i contains a connected component S with more than one element, then there 
exist 1 < V < V-i < ■ ■ ■ ii < m < ki < k2 < ■ ■ ■ < kr such that 



Ulki Wife 



■Wlfe,_i Wife, 



Wi2fe2 



> or < 



Ui2k2 



I Wiifei 



are the vertices of S. 
A matrix u G U^rn) is a path representative if 

(a) At most one connected component of Gu-i has more than one element, 

(b) If Gu-i contains a connected component S with more than one element, then there 
exist 1 < ir' < ir'-i < ■ ■ ■ < ii < m < ki < k2 < ■ ■ ■ < kr with r' e {r, r — 1} such that 



Mlfei 



Uirkr 



'^ir-lkr 



are the vertices of S. 
Let 



or 



Mlfei 



Ui2k2 



'^ir-lkr- 
Wj, — 2kr- 



'^(m) = {w G C/(m) I u a comb representative} 
■^{m) = {" ^ U(rn) I w ^ path representative}. 



If « e Zy s has a connected component Su with a vertex in the first row, then we can 
order the vertices of Su by starting with the vertex in the first row and then numbering in 
order along the path. For example. 



\ 



Xk-2::::-Xk-l 
Xk-3 



I 



where the indices of the x's indicate the prescribed order. The baggage of Su at Xj is 

bag(a;,) = a;,-(-a.,-_i)-^a;,_2(-x,-3)-^ • • • {{,-\y^^ x^i-^y^\ (3.1) 
Proposition 3.1. Let < m < n. Then 
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(a) 7^^^ is a set of superclass representatives for 

(b) ^'(jn) of superclass representatives for 11^^)- 

Proof, (a) First note that the above representatives are in different superclasses of f/„, since 
the corresponding representative would have Miifci,'Ui2fe2 5 • • • j'^vfcr ^ only nonzero ele- 
ments in their columns (by row reducing up). 

Since 1 is incomparable to j G {2,3,..., m} in ■P(m), we may not add row j to row 1 if 
j < m when computing superclasses. Thus, the orbits arc the same as the corresponding 
orbits in U„ except for the first row, and every column can have at most two nonzero entries 
with one of those entries being in the first row. Therefore it suffices to show that if the rows 
of the second nonzero entries do not decrease as we move from left to right, wc can convert 
them into an appropriate form. The following sequence of row and column operations effects 
such an adjustment. 



Ulk 




Ulk 








Uji 



Ulk 







(b) Note that row and column operations imply that 



\ X2 Xs 



\ 



Xk 

Xk-2 Xk-1 
Xk-3 



( XX bag(x3) 



and 



x^ 



\ X2 



are in the same superclass. Thus, (b) follows from (a). 







Uik -UjkU-^f. Ull 



U-jl 



Uik 



Ujl 



bag(xfc-3) bag(a;fe_i) \ 

Xk 

Xk-2 



J 
(3.2) 

□ 



3.2 Superchciracter representatives 

Recall that we identify A € with matrices A e n-p by the convention 

Xij = A(e,j), where e n has {i,j) entry 1 and zeroes elsewhere. 

A function A e n*^^^^ is a comb representative if 

(a) At most one connected component of Gx has more than one element, 

(b) If Gx has a connected component S with more than one element, then there exist 
ki > k2 > ■ ■ ■ > kr > m > ir' > ir'-i > ■ ■ ■ > ii > I with r' e {r, r — 1} such that 



Alfel 



K 



- 1 fcr 



K 

A, 



irki 



or 



^i2k3 



^iik2 



Alfci 
^hki 



A^ 



-ifei 



are the vertices of S. 
A function A G n*^^ is a path representative if 
(a) At most one connected component of Gx has more than one element, 
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(b) If G\ contains a connected component S with more than one element, then there exist 



ki > k2 > ■ ■ ■ > kr > m > ir> > ir'-i > ■ ■ ■ > ii > I with r' € {r,r — 1} such that 



Xi. 



> or < 



^^if kj' 



-ikr- 



Xiik2 
Xi2k2 



Xiiki 



are the vertices of S. 
Let 



A a comb representative} 
A a path representative}. 



If A G -2(m) has a connected component S\ with a vertex in the first row, then we can 
order the vertices of S\ by starting with the vertex in the first row and then numbering in 
order along the path. For example, 



Vr-2 

Vr-yJ-i 



ys—m 
2)4 



(3.3) 



where the indices of the y's indicate the prescribed order. The baggage of S\ at yj is 

bag(%) = yj{-yj.^)-'yj.2{-yj-s)-' ■ ■ ■ 

Proposition 3.2. Let < m < n. Then 

(a) T^jn-j is a set of supercharacter representatives, 

(b) Z(^jn) is a set of supercharacter representatives. 

Proof, (a) First note that the above representatives are in different J7„-orbits, since the 
corresponding representative would have Ajife^, Ajj^j, . . . ,\i^kr as the only nonzero elements 
in their rows (by row reducing down from the first row). 

We may assume the first row has at most one nonzero entry by column reducing. If it 
has no nonzero entry, then further reductions are the same as in [/„. If the first row has a 
nonzero entry in column I, any row j with 1 < j < m can have at most two nonzero entries, 
since we can column reduce using every column except column I. In fact, the only way row 
j can have two nonzero entries is if one is in column I and the other is to the left of I. Note 
that 



Aife 
Xjk' 



Xji 



-A^j^ AjiRow(i)+Row(j) 



Aife Xii 

—Xik'X^i Xji Xjk 



A-;, A- 1 Xj, Col(fc')+Col(fc) 



Xik 







Xjk 



Xii 




Thus, the second nonzero entries can be arranged to occur in decreasing rows from left to 
right. 
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It suffices to show that if a row has two nonzero entries then both columns of the nonzero 
entries must be to the right of the mth column. Note that if fc, k' < to, then 



Xik' ^ii 
Xjk Xji 



X7i^] X^l\J^^ \jkCo\{k')+Col{k) 
— XT'^^Xji Row(i)+Row(j) 











Xik' 



-A-iA.fc,Col(/)+Col(fc') 





-A,fe,A- 







Xik' 


X.a 







Xii 


\ 




) 








Xii 




Xii 


X]k 







Xu 




Xii 


X,k 






Thus, any nonzero pair of nonzero entries in a row must occur to the right of column m 
(b) Note that row and column operations imply that 



yk-3 

Uk-i yk-2 

\ Vk 



and 



yiba.g{y2) 



yk-3 



yk-1 



\ 



-yi^^g{yk-i) 

-yibag(?;fe_2) 
-yibag(?/fe) / 



are give rise to the same supercharacter, so (b) follows from (a). 



(3.4) 
□ 



4 Supercharacter formulas for U(^rn) 

This section develops supercharacter formulas for both comb and path representatives. After 
developing tools that allow us to decompose characters as products of simpler characters, we 
prove a character formula for comb characters. We then use the translation between comb 
and path representatives of (|3.2p and p.4p to get a more combinatorial character formula 
for path representatives. 



4.1 Multiplicativity of supercharacter formulas 

Let u e J7(m)- For a connected component 5* of Gu-i, let u[S] G C/(m) be given by 



u[S]jk 



Ujk, if Ujk e Vs, 
0, otherwise. 



Similarly, let A G n^^y For a connected component T of Ga, let A[r] G n*^^.j be given by 

XiThk = 



Xjk, if Xjk G Vt, 
0, otherwise. 

The following lemma allows us to decompose the supercharacter formula of a pattern 
group U-p by connected components. 

Lemma 4.1. Let u G U-p and A G n^. Let Si, 82, ■ ■ ■ , Sk be the connected components of 
Gu-i and Ti, T2, ■ ■ ■ ,Ti he the connected components of G\. Then 



9 



Proof. Let U = U'p. The proof follows from the following two claims: 

(1) If A has two connected components S and T, then 

(2) If u has two connected components S and 5', then 

(1) Note that since T and T' involve distinct rows and columns, the left orbits of A[T] and 
A[T'] are independent and involve distinct rows. Tims. 

|f/A| = |C/A[T]||f/A[T']|. 

In fact, for A' e UXU, 

Thus, by definition 

' ' \'eu\u 

E e(-7(«-i)-M«-i)) 



\UX[T]U\\UX[T']U, 

/»et/A[T']17 

|C/A[T]||C/A[T']| 



|;7A[T]f/||f/A[T']C/| ^^^^^^^^ 

/ieC/A[T'](7 



E 0(-7(«-l))eHi(u-l)) 



(2) For any u' € UuU, 

\{{v,w)€{Uu[S]U)x{Uu[S']U) I u'-l^v-l + w-l}\= 1^"^ I^^N^I . 
We have that 

ml=(7u[S']!7 

_,,^. x\n[S])x\u[S']) 

^ ^ > x^(i) x^(i) ' 

as desired. □ 
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Corollary 4.1. Let u G U-p and A £ n^, with connected components Ti, . . . , T;. Then 

i=l 

To obtain character formulas for [/(„i) we will require a slightly more refined multiplica- 
tivity result that depends on the poset structure 'P(m) and a choice of comb representatives. 
For u £ ?7(m) and 1 < fc < n, let u[k] g C/(„i) be given by 



u[k\ 



0, otherwise. 



That is, u[k\ is the same as u in the fcth column, but zero elsewhere. For A G n^, let 
A[m, fc] G Hp be given by 



'^ii, if Z > fc and ^ for some j > i, 
0, otherwise. 



That is, A[u, fc] is the same as A weakly NorthEast of the nonzero entries of u in the fcth 
column, but has zeroes elsewhere. 

The following lemma states that we can compute supercharacter formulas for column 
by column on the superclasses. 

Lemma 4.2. Let u G ?7(m) with u G o.'^^d let A G 7(„i). Then 

(a) The character x'^(w) ^ and only if x^^^'^\u[k]) ^ for all 2 < k < n. 
(h) The character value 

y.HnM{u[k]) 



X\u)=x\l)\{ 



k=2 



Proof, (a) Let M correspond to (A,w) as in Theorem l2.1l Note that Af(jj) (j, ;) , M(j ;/) G 
implies Xa,Ujk, Xti' ,Ujk' G F^, so 



k k' I I' 



] Ujk Ujk' 



and 



However, since u G the only row of 7i which can have more than one nonzero entry is 

row 1. Since i < j, we have fc = fc' and the nonzero entries of u contribute to distinct rows 
of M. Similiarly, if M(i^j)^(^k.i), ^(i' .j')Xkd) ^ implies XiuUjk,K'uUj>k G F^, so 



and 



j I Ujk I i' I Xi'l 

j' \ Ujik 

Thus, distinct columns of u contribute to distinct columns of M . For 1 < fc < n, 

^ rows of A/ that have nonzero entries corresponding 
to the nonzero entries of u in column fc 

^ _ columns of M that have nonzero entries corresponding 
^ to the nonzero entries of u in column fc 



(4.1) 



By choosing an appropriate order on the rows and columns of M, 

M = Mr.^c^ ® Mr,^c2 © ■ ■ • ® Mfl„,c„ , (4.2) 
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where M^^^Ck the submatrix of M using rows Rk and columns Ck- 

Using ()4.2|) . there exists a solution to Mx — ~a if and only if for each 1 < fc < n, there 

exist Xk e F^'' such that MR^^Ck^k = -aR^- 

If aij 7^ 0, then there exist Xik, Ujk G for some k. Since row j in u has at most one 

nonzero entry, = Ujk^ik ■ Thus, o/j^ only depends on the pair (A[u, fc], M[fc]). 
By (|4?2)) . we have 

NuU(A/) = Null(Mi^,,cJ © NuU(Mfl,,cJ © • • • © NuU(Mfl„,c„), 

so 6 is perpendicular to Null(Af) if and only if hcf, is perpendicular to Mii^.^Ck ^'^^ ^- The 
condition {k,l) G Ck implies Ujk for some j, so bki G F^ implies bki — uifeAi/ + UjkXji- 
Thus, only depends on the pair (A[u, fc], M[fc]), and (a) follows, 
(b) Since Ci = i?i = 0, it follows from ((421) that 



rank(M) = ^rank(Mfl„cJ = ^ rank(M;^,,cJ 
It follows from (a) that 



k=l k=2 



e{x-b) = l[9ixck-bck), 



k=2 



and by inspection 



0ox{u- 1)^11 e{u,kX,k) - n n ^i^jkXjk) = n ^(^i^' ^k^w - 1))- 

(j,k) k=lj<k k=l 

Now (b) follows from (a). □ 

Remark. This lemma depends on the choice of representatives. In particular, it is not true 
for path representatives. 

4.2 A character formula for comb representatives 



It follows from Lemmas 14.11 and | 4 . 2 1 that to give the character value of we may assume 

u — 1 has nonzero entries in one column and Gx has one connected component S. 

Theorem 4.1. Let u G C/(m) such that u G '^(m) '^^'^ u — I has support supp(u — 1) C 
{j,k)}. Let X G 7(„i) be such that X has one connected component S with Cols(S') = 
{h < h < ■ ■ ■ < Is}- Then 

(a) Let ii > 12 > ■ ■ ■ > ig-i be such that Xij^ ^ 0. The character degree 

s-l 

-Q qU-ia-i^ ^fXu, = for all i > h, 

s-l 

gL-i-i qi-d-^d-i^ if Xii^ ^ for some i > ii. 



(b) The character 

X\u) = 

unless at least one of the following occurs 

(1) UjkXik ^ implies i — I with j < m or i > j; and uikXn +UjkXji — for all j < k < I, 

(2) UjkXik ^ for some 1 < i < j < m, but \Rk\ — \Ck\ > (Rk and Ck are as in J^.Jp j, 
(S) uikXii^ + UjkXji 7^ for some m < k < I, but Xik — for all i and \Rk\ > \Ck\ > 0, 
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(4) Ujk, ^ji^ , Xii^ G with i < j' < k < Is with Xjk' = for all k < k' < I 
(c) The character values are 



X\u) 



if (2) or (3) or (4) 
K^e{-X7i]\,k[uiuXii^ + UjfeAjij), if (2) and (4), 



qlC 
X 



where Src = 1 «/ |Cfe| > \Rk\ and 6b.c = i/ |Cfe| < \Rk\. 
Proof, (a) This is just a statement of the fact that 



combined with the structm'c of S. 
(b) and (c). Note that by Lemma 



A[«, 



so we may assume M = Mji^^Ck (see (|4.2p ). Let Rows(S') {ii > . . . > Zs} or Rows(5') = 
{io > ii > . . . > is} he the rows with nonzero entries in 5* such that Xi^i^, Xi^i^ £ , and, if 
io S Rows(5), then Xi^i^ ^ (see the definition of comb representatives in Section [321) ■ Let 
r and r' be minimal such that Ir > k and < j. Then 



M = 



UjkXi^i^ 



S'ujkXii^ \ 
UjkXi^i^ 

UjkKr^^ll, 



\ 



where S' — 



1, if j > m, 
0, if j < ni 



UjkK^lls I 



Thus, the rank of M is glC'=l-'5. 



(4.3) 



Furthermore, a £ Fjj and 6 G Fjj are given by 



a 



bki 



UjkXik, 

uikXii + UjkXji, if I G Cols(5), 
otherwise. 



for (ij) e Rk, 
for (fc,OeCfc. 



If a = then AI ■ — —0 is easily satisfied, and if 6 = then b ± Null(Af) is also trivially 
satisfied. Thus, x'^(u) ^ if UjkXik = for all i < j < k in 7^(m) and MifcAi; + UjkXji — for 
all 1 < j < A: < L Note that in the poset P(m), 1 ^ j for j < m. 

Suppose Oij ^ 0. Note that Mx = —a can only be satisfied if row {i,j) of M has a 
nonzero element. That is, there exists i < j < k < I such that A^; / 0. Consequently, we 
may assume k < Is- If j > m, then 5 = 1, so {Mx)ij ^ if and only if {Mx)ij ^ 0. However, 
aij ^ and {Mx)ij ^ implies the first row of A has two nonzero elements, contradicting 
the structure of S. Thus, if aij ^ and Mx = —a for some x, then j < m and k < Is- 

Suppose Oij — UjkXik with j < m and k < Is. By (14. 3|) . (i,fc) = (ir-i,^r-i)- Note 
that {Mx)ij 7^ if and only if {Mx)i ,j ^ 0. Since Ujk' — for all k' ^ fc, in this case 
r' = r — 1 or \Ck\ — \Rk\- If we choose x such that 



Xki = { X,]_X 



ik J 



if / - Is, 
if / = Id, 
otherwise. 



where {k, I) G Ck 
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then Mx = —a. 

If bki ^ and M has no nonzero entry in column {k,l), then b is not perpendicular to 
Null(M). Thus, if bki ^ we must have Xji, Xa e with i < j. In particular, j < m, and 
either uik = or u has two nonzero elements. Since only the last column of S can have more 
than one nonzero entry, I = 1^, and bki^ = uikXu^ + UjkXji^. Note that 



dim(NuU(M)) 



s — r, if S' = 0, = r, 
0, otherwise. 



It follows that when 6^0, then b is perpendicular to Null(M) if and only if r' > r if and 
only if \Rk\ > \Ck\ (if S' = 1, then j > m). 

In the case that j = ir-2 and k = lr~i, we have 



9{x ■ b) = 0[-X^i'^^XikiuikXu^ + UjkXjiJ), 
Otherwise, 9{x ■ b) — 1. 



where i = ir 



□ 



At this point, it may be helpful to give a more visual interpretation of the conditions in 
Theorem 14. II by considering the configurations of superimposed graphs Gx and Gu- Recall, 
for A G -Z(-„j-) U T{m) there is at most one connected component of Gx that can have more 
than one element (or can have a vertex in the first row of A). Therefore, for A G -Z(-„j) U7(„-), 
let 



lc(A) : 
br(A) : 

Wt(A) : 

For example, if 



Sx = the connected component of Gx that has a vertex in the first row 

minjfc I Sx has a vertex in column fc}, if S'^ 7^ 0, 
0, otherwise, 

max{j I Sx has a vertex in row j}, if Sx ^ 9, 
n, otherwise. 



^(Nonzero entries in row br(A) of A) — 1, if S'a ^ 



0, 



/00000a\ 
c 6 
e 
d 


\ooooooy 



otherwise. 



then Sx — 



lc(A) 5, 
br(A) = 4, 
wt(A) = 0. 



In the following discusion, we will suppress the values of the vertices and distinguish 
between Gu and Gx by the following conventions. 





Gu-i 


Gx 


Vertices 


• 


■ 


Edges 


• • 


■ — ■ 



If \Sx\ > 1 and wt(A) = 0, then add an edge to the non-zero vertex of row br(A) that extends 
West of this vertex. 



thereby "completing" the comb. 
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Vertices of G„ see North in their column and East in their row, while vertices of Gx see 
South in their column and West in their row (in both cases they do not see the location they 
are in). That is, ^ 

I ^ and |. 

Connected component S of Gu and T of Gx see one-another if when one superimposes their 
matrices, a vertex of S sees a vertex of T (and vice- versa). 

The tines of Sx are the pairs of horizontal edges with their leftmost vertices. For example, 
the tines of 



Suppose u S Ufja) has at most two nonzero superdiagonal entries uik,Ujk G F^, for some 
1 < k < n, and suppose A £ ^*{m) such that Gx has exactly one connected component S. 
Then column fc of u is comb compatible with S if the following conditions are satisfied. 

(CCl) If column k oi u has exactly one nonzero entry Ujk in column k and S ^ Sx, then Ujk 
cannot see S, 



compatible not compatible 

(CC2) If uikjUjk G F^, with 1 < j and S ^ Sx, then S cannot see Uik or Ujk, 



■ • 



compatible 



not compatible 



(CCS) If column k of u has exactly one nonzero entry Ujk in column k and S = Sx, then u 



jk 



sees S if and only if j < to, Ujk is South of the end of a tine and weakly North of the 
next tine to the South (if there is another tine). 



compatible 



not compatible 



(CC4) If uik,Ujk G F^ , with 1 < j and S — Sx, then S sees uik or Ujk if and only if either 
Ujk is not South of the end of a tine but on a tine of S, or Ujk is South of the end of a 
tine and weakly North of the next tine to the South, 



1 • 1 
1 ■— 1 
• 

1' ■ 1 


1 • 1 
1 ■— 1 

1' ■ #-1 


f ( • f 


1 ■— 1 
• 

1' ■ 1 


1 1 1 


1 1 1 


1 1 1 



m • m 

B-ii 



compatible not compatible 

From this point of view. Theorem 14.11 translates to the following corollary. 
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Corollary 4.2. Suppose u G ?7(m) has at most two nonzero superdiagonal entries uik,Ujk S 
¥q, for some 1 < k < n. For A G ^m); suppose G\ has one connected component S. Then 

(a) The character degree 

A/-|\ f |{* < i e 'P{,n) I ^ik ^ 0, for k > j > i > 1, \ik' ^ implies k' > k}\, z/wt(A) = 0, 
^ ^ ^ \ \{i<j e ■P(„) I A,fc ^ 0, for k>j>i, X,k' ^ implies k' > k}\, i/wt(A) = 1. 

f6j T/ie character 

X\u) = 

unless column k of u and S are comb compatible and in condition (CC4) if uik sees S and 
Ujk is not strictly South and weakly East of the end of a tine, 

Ulk At/ 

, then uikXii + Ujk^ii = 0- (4.4) 

■ Ujk A,i 

, I 



(c) IfxHu)^0, then 



qc(u,x) 



n 

i<j<l 



^ikiuik^ll + UjkXjl)) 



where 



c{u, A) 



\{l > k I Aji 7^ 0, for some i < j}\, 
\{l > k I Ai; 7^ 0, for some i < j}\, 
\{l > k I Xii 0, for some i < j}\ 



if Ujk ^0, j > m, 

if Ujk, XijiXii e with j < m, i < j' < k < I, 
1 , otherwise. 



4.3 A character formula for path representatives 

For A G U let S\, lc(A), br(A), and wt(A) be as in the previous section. For 

A e order the vertices of S\ starting with the vertex in the first row and proceeding 

along the path to the vertex in position (br(A), lc(A)). For example, if 



/00000a\ 
c 6 
e 
d 


yoooooo/ 



then ^A 



lc(A) = 5, 
br(A) = 4, 
wt(A) = 0. 



Before translating from comb representatives to path representatives, we will add some 
decorations to the graphs Gu-i and G\ of Section [3l We will again suppress the values of 
the vertices and distinguish between G,j_i and Gx by the following conventions, 





Gu-i 


G\ 


Vertices 


• 


■ 


Edges 


• :::::• 


■ ■ 



If 5*;^ ^ 0, then add an additional edge from the vertex in position (br(A), lc(A)). If 
wt(A) ~ and \Sx\ 7^ 1 extend the edge West until it reaches just pasat the (m + l)th 
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column. If wt(A) = 1 or \S\\ = 1, then extend the edge South until just past the mth row. 

For example, 



/ 1 



V 



I 1 



1 / 



V 



\ 



, or 



I 



1 / 



A bottom corner of A is a vertex v in Sx with a horizontal edge extending West of v. A top 
corner of A is a vertex v in S\ with a vertical edge extending South of v. All vertices of G\ 
which are not in Sx are considered to be both top and bottom corners. 

Similarly, if u e ^(m)' t^i^n G„_i has at most one connected component Su that has a 
vertex in the first row. Order the vertices of Su starting with the vertex in the first row, and 
proceeding along the path. If |S'„| > 1, add an edge to Su by extending an edge from the last 
vertex in the opposite direction of the previous edge (either East or North) . Furthermore, 
if l^ul > 1, then view the first edge as not being in the same plane as the matrix, so it no 
longer crosses any edges that are in the plane of the matrix. Thus, Su will be of the form. 



V 



1 






• ::::::[ 






1 V. 










1 









1 



or 



1 












1 ■.; 


*■■••' 




.«::• 




1 









1 



The left corners of u are the leftmost nonzero entries in the rows of m — 1. The right corners 
of u are the rightmost nonzero entries in the rows of u — 1. 

Left and right corners see North in their column and East in their row, while top and 
bottom corners see South in their column and West in their row (in both cases they do not 
see the location they are in). That is, 



and 



Connected components S of G„ and T of Gx see one-another if when one superimposes their 
matrices, a corner of 5* sees a corner of T. 



Fix 



and A € Z(^^) with a connected component S of G„ and T oi Gx- The 



components S and T are path compatible if the following conditions are satisfied. 
(PCI) liSi^Su and T ^ Sx, then S cannot see T, 



compatible 



not compatible 



(PC2) If S' = S'„ and T ^ Sx, then S sees T if and only if T touches a vertical edge of S and 
no left corner of S sees T. 



• :::| 



B:::| 



• :::| 



::• 



■■.■■■<e*IE) 



*:::| 



compatible 



not compatible 
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(PCS) If 5 ^ Su and T ^ S\, then S sees T if and only if S touches a vertical edge of T and 
no bottom corner of T sees S. 



■J- 



compatible 



not compatible 



(PC4) If = S'„ and T = Sx, then S sees T if and only if T is never strictly South of S; S 
ends weakly East of the beginning of T; and left corners of S and bottom corners of T 
only see one-another horizontally. 




compatiblc 



*:::::::•. 
*::::::::::::» 



not compatible 

Note that (PC1)-(PC4) are translations of (CC1)-(CC4) via the correspondence (p^ . 
Corollary 4.3. Let u e C/(m) he such that w — 1 G ^^m)' '^"''^ ^ Then 
(a) The character 

X\u) = 

unless the connected components of Gu-i and G\ are pairwise path compatible, and in 
the superimposed matrices, every 



fe ::::::::::::> 



implies hag{xj)ha,g{yk) = 1- 



(4.5) 



(b) The character degree 

x,^s ( \{i < j <E 7^(„i) I Kk is a bottom corner for some k > j/wt(A) = 0, 

|{i < j G 'P(m) I \k is a top corner for some k > j}\, i/wt(A) = 1. 



(c) IfxHu)^0, then 



1 



, bottom corners Xn \ 

■Jt corners ^) \ Vk or UX 

"j* ^ with i < j < k < I j ^ 



Yl 6'(bag(a;j)bag(2/fe)). 



Proof. This corollary follows directly from Corollary 14.21 with the following observations, 
using (fO]) . 
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(a) If a bottom corner of T sees a left corner of S horizontally, then we arc in the situation 
of dill), so 

J/l bag(£!;3_i)::::::::::: yi 



J!:: :::^ ^ 



-j/ibag(i;fc) 



SO the comb representations of A and u must satisfy condition (|4.4p . However, this is equiv- 
alent to bag(xj)bag(yfc) = 1. Thus, Corollary [42] (a) is satisfied if and only if Corollarv l4.3 
(a) is satisfied. 

(b) is straight-forward translation of the combinatorics. 

(c) First note that 



n 



bottom corners A^; 
with i < j < k < I 



n 



1 



c{u.X) 



If x'^(w) ^ 0, and we have no configurations of the form 



::::::l/lbag(2;j) 
. 1 



yi 

Vk — ■ 



— i/lbag(afc_i) 
— yihag{yk+i) 



(4.6) 



(4.7) 



then both character formulas are equal. If (|4.6|) occurs then both the path character formula 
and the comb character formula get a factor of 

0{hag{xjyi)). 

If (|4.7|) occurs then the path character formula gets a factor 

0{yk+ix-i+i)9{ha,g{xj)ha.g{yk)), 

and the comb character formula gets a factor of 

{-{-yihag{yk-ir^yk{hag{xj)yi + Xj+i{-yihag{yk+i))) , 

However, a simple computation shows that these two quantities are equal. Thus, the char- 
acter formulas for the two types of representatives are equal. □ 



4.4 Example 

Let n — 7, m ~ 4:, 



/lOOOaOOX 



10 
1 e 
1 







d 


b c 



1 
1 

Vooooooiy 



and 



A = 



/000000a;\ 
y 

z 



\oooooooy 
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so the decorated superimposed matrix of u — 1 and A is 













'a 







\ 













-9- 

















e ■ 





p 



















:te: 


::C 









































































/ 



All the connected components are pairwise compatible, 



: : ; 



:: 7 
;.fe:::::C 



Thus, 



X\u) = 



if and only if hag{d)h&g{y) = 1 if and only if d{—c ^)b{—a ^)y{—x ^) = 1. If x^iu) 7^ 0, 
then 

1 



1. 



5 Interpolating from Un to Un-i 



This section uses a restriction rule from 1) to [/(„) for supercharacters to deduce a 
restriction rule from C/„ to Un-i- 



5.1 A restriction rule for U(^rn) 

Note that if A £ -Z(„j_x): then A G -H(„j) unless lc(A) = m. Let A^ G ^(m) be given by 



A^ = 



A, if lc(A) ^ m, 

A — AimGim if lc(A) — m and i is minimal so \im 7^ 0. 



Then A^ is the path orbit representative of t/(„)At/(m). 

Similarly, if A G then A G unless lc(A) = m. Let A^ G T(^m) be given by 

r A, if lc(A) ^ m, 

A^ = < A - \imeim if lc(A) = m, wt(A) = 1, and Aim G S\, 

[ A - AjmCim + Ajmejm - AjiCj;, if lc(A) = m, wt(A) = 0, Xim,X]i G 'S'a with j = br(A). 

Then A-'- is the comb orbit representative of ?7(m)'^C^(m)- 
Theorem 5.1. Let A G and fc = lc(A). Then 



Res 



Ex 



A^+te 



if k = m or wt(A) = 0, 

if k > m, wt(A) = 1, Xmj ^ for some j > k, ^-^^-^ 
% if k > m, wt(A) — 1, Xmj — for all j > k. 



teF„ 



Proof. Using Lemma |4.1[ we may assume that Gu-i has one connected component. We will 
use Theorem 14.31 and Theorem 14.11 to compare the appropriate character values. We split 
the proof into three cases. 
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Case 1. The element u G satisfies Umk' = for all k' > m. 
First, note by inspection that 

X^^l), if fc TO or wt(A) = 0, 

Res'^'"""(Y^)(l) = I if fc > TO, wt(A) = 1, Xmj ^ for some j > k, 

> ^ X^'+*'""(l), if fc > TO, wt(A) = 1, A„y = for all j > fc. 

tSF, 

Since by assumption row m of m — 1 is zero, the fact that 1 ^ m does not affect the 
value of x'^(")- Thus, the only problematic case is the case lc(A) = to. Here observe that if 
AJ- = A - XimSim, then Ui,n ^ if and only if x^{u) = = (")■ 

Case 2. The element u G J7(m) satisfies Umw 7^ for some fc' > to,, and uik' = Umj = for 
all k' > j > TO. 

Note that if fc = m or wt(A) — 0, then 

Thus, it suffices to consider the cases where lc(A) > to and wt(A) = 1. 

Suppose lc(A) > TO and wt(A) = 1 with A„y ^ for some j > lc(A). If fc' < j, then 
X^{u) = = qx^^{u). If fc' > j, then x^l"") = QX^\u) follows from the proof of Claim 1. 

Suppose fc = lc(A) > TO and wt(A) — 1 with Xmj — for all j > k. If fc > fc', then 



tSF, 

since the other summands all contradict (PCI). Thus, in this case, 

tSF, 

If fc < fc', then = x^{u) for all t € F,, so 



If fc = fc', then = 0, by (PCI). On the other hand, 



„ yj. J corners strictly I ^ 

(7 L NE of (m,k') ) 



as desired. 

Case 3. The element u G satisfies Uife',Umfc' £ /o?' some fc'. 

For Case 3 we translate to comb representatives and use Theorem 14.11 For comb rep- 
resentatives, we may assume that uik',Umk' are the only nonzero entries in u — 1. Define 

G C/(m-i) by ~ u — uik'Cik'- Note that is the representative for the superclass 
containing u in C/(m-i)- Thus, we will show 

[uM'] (y)^ if = TO or wt(A) = 0, 

Res'^''""''(Y^t"^''''l)(uf) = I if ^ > wt(A) = 1, Xmj ^ for some j > 

" ' ^ ^^An«,fc']+te,„,(^)^ iffc>TO,Wt(A) = l, Au^O, A„,,=0,J> 
tSF, 

In the cases wt(A) = or fc = to, the character value ^'^'"^''^ '(w^) = if and only if at least 
one of the following conditions hold 
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(a) Xij 7^ for 1 < j < k', 

(b) A,„; ^ for some I > k', 

if and only if ^^^^'^''^ 1 (u) — Q. The restrictions for these cases follow. 

Suppose lc(A) > m, wt(A) = 1 and Xmj for some j > k = lc(A). If j > k' , then 
^A[«T,fc'](-yT) ^ If j < k', then k < k' , and (^T) ^ q = x^^^"'''''^ 

by a similar argument as in the previous cases. 

Suppose lc(A) > m wt(A) — 1 and Xmj = for all j > k. Let I be such that Xu ^ 0. If 

; < fc', then ;(;^H«.fc']+te„,,(y) ^ ^AiKfe'](-y) ^ ^-l^A[«T,fc'] ^-^^^ ^ ^ ^j^g^ ^A[«T,fc']|.^^ ^ 
= T^Ai[«,A;']+te„,, ^^^^^ 

Suppose / > k' . If Ajfc/ 7^ for some 1 < j < k' , then = 0. On the other 
hand, by Theorem 14. II 

E x''["''='l+*^"'M = + E Ku^O{^^Ji'^sk'iu,,^Xu+Umk't)) = 0. 



teF„ 



If Xjk' = for all j < fc', then 



since, according to condition (1) of Theorem l4.11 all the other summands are zero. However, 
^Ai[«,fc']-«i,,An«-t,e„, ^ as desired. □ 

5.2 A restriction rule for Un 

For A e Sn{q), and «, A: G Z>i, let 

'A, if i = fc, 

qX {fc}, if Xii 7^ for some I > k, 

•^L.fc^o "L^}' if Ajfe 7^ 0, 

A *i {fc} = < A {k} + A| A,^=t {j}, if Ai, 7^ for some i < j < k, (5.1) 



A*m{fc}+ E ^Ia.,= 



if Aij = for all j > i. 



We will extend this product to supercharacters by the convention 



Corollary 5.1. For A G 5„(q), 

Res^:_,(x') = { Ja^ 
Proof. Note that for 1 < m < n. Theorem 15. II implies 



=ic;^ if Xik 0, for some 1< k, 

otherwise. 



x^, if lc(A) 7^ m and wt(A) = 0, 

X^-^™^"", iflc(A)=m, 

(X-^) = ■( qx^, if lc(A) > m, wt(A) = 1, A„y 7^ for some j > lc(A), 

E x^^*"'"' , if A; = lc(A) > m, wt(A) = 1, Xmj = 0, for aU j > k. 

The multiplication given by (|5.ip is an iterative version of the restrictions from to 
U(^m) ? where the last two cases in (jS.ip correspond to the last case in the restriction, depending 
on whether there exists Xmj G for some j < lc(A). □ 
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5.3 Examples 

Example 1. Consider the case q ^ 2. Then we may choose our representatives A G n* and 
u G Un to correspond to set partitions of {1, 2, . . . , n}. For example, if 

A = {1--5 I 2'~.6 I 3--4}, 

then 

Res^:(x^) = X^^-~'l^l^^*i{5} 

= ^{l|5|2-€|3^} ^2 {5} 
^ 2;^{l|5|2-6|3-4} 

= 2x{i|'l^l^} *4 {5} + 2x^i|^l^l*> *4 {4} 

= 2x^'l'l^l^^ *5 {5} + 2x^^l^l^^> *5 {5} + 2x^i|^l^|4> 

^ 2x^^'^'^'^^ + 2x^^'^'^*'~^^ + 2x^^'^'^'^^ 



Note that the final result gives the representative of Un-i as a submatrix of C/„ (since by 
construction the first row will always be zero). To obtain a set partition of rt — 1, we would 
remove 1 and renumber the rest of the entries j j — 1- 

Alternatively, this algorithm may be viewed as a "bumping algorithm" , where we replace 
the 1 by all other "possibilities," suitably defined. 

Example 2. Linear characters of Un are those characters whose superclass representative 
satisfies, i and j are in the same part only if i + 1, « + 2, . . . , j — 1 are also in the part. In 
this case, 

X^*l{J}-X^ 
Example 3. On the opposite extreme with have the case 

A = I 2 I 3 I •■ • I n- 1}. 

In this case, 

f'es„(<j), 

^i3=0,l<i<i<n 



5.4 An alternate embedding of Un-i 

The paper jT^ uses a different embedding of C/„_i into Un (obtained by removing the last 
column rather than the first row). This alternate embedding gives a different restriction rule. 
For G Sn{q) and j,l G Z>i, let 



{j} *i M 



{j}*'-im|u,.,=o' 



if J=^, 

if there is i < j with 7^ 0, 
if f^ji 7^ 0, 

{j} *i-i M + {^} m| Mfc,=o , if there is k > j with fiki 7^ 0, 

tew^ 

otherwise. 



q{{j} fi) 



tew: 



Then by symmetry arguments from [13j . we obtain the following corollary for this alternate 
embedding of Un-i into C/„. 
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Corollary 5.2. For A G Sn{q), 

c/„_iVA ^ otherwise. 

In particular, unlike in the symmetric group representation theory, the decomposition of 
induced characters depends on the embedding of f7„_i into Un- 
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